Methods and systems for compressing a video stream with minimal loss after subsampled decoding

ABSTRACT

Transcoding of a video stream to reduce the size of the video stream with little, if any, loss in video quality after subsampling. After accessing a video stream of video pictures (i.e., video frames or fields), the blocks of the video picture are each subject to matrix pre-multiplication and post-multiplication. Such matrix multiplication does degrade the video quality if subsampling was not to occur. However, the pre-multiplication and post-multiplication matrices are calculated based on the subsampling matrices that will be used to ultimately subsample the video stream such that after subsampling eventually occurs, the matrix multiplications result in minimal loss of video quality.

CROSS-REFERENCE TO RELATED APPLICATIONS

The present application is a continuation application ofcommonly-assigned U.S. patent application Ser. No. 09/886,693 filed Jun.18, 2001, entitled “Methods and Systems for Compressing a Video Streamwith Minimal Loss After Subsampled Decoding” and which is incorporatedherein by reference.

BACKGROUND OF THE INVENTION

1. The Field of the Invention

The present invention relates to the field of video processing. Inparticular, the present invention relates to the compression of a videostream when it is known that the video stream is to be subsampled forminimal loss after subsampled decoding.

2. Background and Related Art

Video constitutes a series of images that, when displayed above acertain rate, gives the illusion to a human viewer that the image ismoving. Video is now a widespread medium for communicating informationwhether it be a television broadcast, a taped program, or the like. Morerecently, digital video has become popular.

An uncompressed digital video stream has high bandwidth and storagerequirements. For example, the raw storage requirement for uncompressedCCIR-601 resolution 4:2:2: serial digital video is approximately 20megabytes per second of video. In addition, associated audio and datachannels also require bandwidth and storage. From a transmissionbandwidth perspective, 20 megabytes per second is much faster thanconventional transmission techniques can practicably support. Inaddition, from a storage perspective, a two-hour movie would occupyapproximately 144 Gigabytes of memory, well above the capabilities of aconventional Digital Versatile Disk (DVD). Therefore, what were desiredwere systems and methods for compressing (or coding) digital video in away that maintains a relatively high degree of fidelity with theoriginal video once uncompressed (or decoded).

One conventional high-quality compression standard is called MPEG-2,which is based on the principle that there is a large degree of visualredundancy in video streams. By removing much of the redundantinformation, the video storage and bandwidth requirements aresignificantly reduced.

FIG. 1A illustrates a display order 100A of a sequence of pictures. Ifthe video stream represents progressive video, the pictures representindividual progressive frames. If the video steam represents interlacedvideo, the pictures represent individual interlaced frames containingtwo fields each.

Under the MPEG-2 standard, there are three classes of pictures,I-pictures, P-pictures and B-pictures. While MPEG-2 allows for a numberof display orders for groups of pictures, the display order illustratedin FIG. 1A is commonly used. In this common display order, there are aseries of I-pictures. For clarity, only I-pictures I1 and I16 are shownin FIG. 1A. Each consecutive I-picture pair has four P-picturesinterspersed there between. For example, P-pictures P4, P7, P10 and P13are interspersed between consecutive I-pictures I1 and I16. In addition,two B-pictures are interspersed between each I-picture and each of itsneighboring P-pictures. Two B-pictures are also interspersed betweeneach consecutive P-picture pair. For example, B-pictures B2 and B3 areinterspersed between I-picture I1 and P-picture B4, B-pictures B5 and B6are interspersed between P-pictures P4 and P7, B-pictures B8 and B9 areinterspersed between P-pictures P7 and P10, B-pictures B11 and B12 areinterspersed between P-pictures P10 and P13, and B-pictures B14 and B15are interspersed between P-picture P13 and I-picture I16.

The I-pictures are “intra-coded” meaning that they can be restructuredwithout reference to any other picture in the video stream.

The P-pictures are “inter-coded” meaning that they may only berestructured with reference to another reference picture. Typically, theP-picture may include motion vectors that represent estimated motionwith respect to the reference picture. The P-picture may bereconstructed using the immediately preceding I-picture or P-picture asa reference. In FIG. 1A, arrows illustrate the predictive relationshipbetween pictures wherein the picture at the head of the arrow indicatesthe predictive picture, and the picture at the tail of the arrowindicates the reference picture used to reconstruct the predictivepicture. For example, the reconstruction of P-picture P7 uses P-pictureP4 as a reference.

B-pictures are also inter-coded. The B-picture is typicallyreconstructed using the immediately preceding I-picture or P-picture asa reference, and the immediately subsequent I-picture or P-picture as areference. For example, the reconstruction of B-picture B14 usesP-picture P13 and I-picture I16 as references.

FIG. 1B illustrates the decode order 100B of the pictures. The decodeorder is similar to the display order except that reference frames aredecoded prior to any predictive pictures that rely on the referencepicture, even if the reference picture is displayed after the predictivepicture. Thus, the arrows in FIG. 1B are all rightward facing.

FIG. 2A illustrates the general process involved with encoding a digitalpicture 201 using an encoder 200A that is compatible with the MPEG-2standard. If the digital picture is to be an I-picture, the digitalpicture bypasses the motion estimator 202 and is provided to thediscrete cosine transformation unit (DCT) 203, which transforms thedigital picture, on a block-by-block basis from a spatial representationof an image to a frequency representation of the image. The frequencyrepresentation is then passed to a quantization unit 204, whichquantizes each frequency, on a macroblock-by-macroblock basis, intodefinable ranges. A “macroblock” is a 16-pixel by 16-pixel array withinthe picture. The quantized image is then passed to a variable lengthcoder 205 which performs, for example, variable length Huffman coding onthe resulting quantized image. The reduced sized I-picture is thenstored or transmitted for subsequent decoding.

If the digital picture 201 is to be a P-picture, the encoding process issimilar as for I-pictures with several notable exceptions. If aP-picture, the digital picture is passed first to the motion estimator202. For each macroblock (i.e., 16×16 pixel array) in the P-picture, themotion estimator 202 finds a close match to the macroblock in thereference picture. The motion estimator 202 then represents themacroblock in the P-picture as a motion vector representing the motionbetween the macroblock in the P-picture and the close match 16×16 pixelarray in the reference picture. In addition to the motion vector, adifference macroblock is calculated representing the difference betweenthe macroblock in the P-picture and the close match 16×16 pixel array inthe reference frame. A macroblock represented as a difference withcorresponding motion vectors is typically smaller than a macroblockrepresented without motion vectors. Discrete cosine transformation andquantization are then performed on just the difference representation ofthe P-picture. Then, the difference information is combined with themotion vectors before variable length coding is performed.

B-pictures are encoded similar to how P-pictures are encoded, exceptthat motion may be estimated with reference to a prior reference pictureand a subsequent reference picture.

FIG. 2B illustrates a conventional decoder 200B in conformance with theMPEG-2 standard. First, a variable length decoder 215 performs, forexample, variable length decoding on the picture. The picture (or thedifference data of the picture if a P-picture or a B-picture) is passedto the inverse quantizor 214 for inverse quantization on amacroblock-by-macroblock basis. Next, an inverse discrete cosinetransformer 213 performs inverse discrete cosine transformation on thefrequency representation of the picture, on a block-by-block basis, toreconstruct the spatial representation of the picture. The spatialrepresentation of the picture is passed to the motion compensator 212where the spatial representation is combined with the motion vectors (ifa P-picture or B-picture) to thereby reconstruct the digital picture201′. The reconstructed digital picture 201′ is labeled differently thanthe original picture 201 to represent that there may be some loss in theencoding process.

In this manner, MPEG-2 combines the functionality of motioncompensation, discrete cosine transformation, quantization, and variablelength coding to significantly reduce the size of a video stream withsome generally acceptable reduction in video quality. Despiteconventional standards such as MPEG-2 that provide significantcompression to a video stream, it is desirable to reduce the bandwidthrequirements of the video stream even more to maximize network andstorage performance.

One way to further reduce the bandwidth requirements is to compress thevideo stream even beyond the compression performed during the originalMPEG-2 encoding processes. However, this results in a loss of videoinformation and thus degrades the quality of the video stream to acertain extent. Therefore, what are desired are systems and methods forfurther compressing a video stream with less, if any, loss of videoinformation.

BRIEF SUMMARY OF THE INVENTION

The present invention extends to both methods and systems fortranscoding a video stream so as to reduce the size of the video streamwith little, if any, degradation of video quality after subsampling. Thevideo steam includes a number of video pictures such as frames or fieldsand may be stored in memory or accessed from a transmission. Inaddition, each video picture includes one or more blocks. These blocksare the fundamental unit upon which subsampling may be performed. Forexample, under the MPEG-2 standard, developed by the Moving PicturesExperts Group, subsampling may be performed on blocks of 8 pixels by 8pixels.

The video management system accesses one of the video pictures from thevideo stream. Then, for at least one block of the video picture, thevideo management system represents the block as a matrix of pixelvalues. Then, the block matrix is pre-multiplied by a pre-multiplicationmatrix and post-multiplied by a post-multiplication matrix. Thepre-multiplication matrix is generated from a subsample matrix thatrepresents the subsampled decoding in one direction. Thepost-multiplication matrix is generated from a subsample matrix thatrepresents the subsampled decoding in a substantially perpendiculardirection.

The pre-multiplication matrix and the post-multiplication matrix arestructured so that the block of pixels is altered in a manner thatsubsampling of the altered block of pixels results in the samesubsampled image as subsampling of the original block of pixels. Thepre-multiplication matrix and the post-multiplication matrix are alsodesigned to decrease the size of the encoded version of the block ofpixels.

This strategic altering of blocks of pixels may be repeated for eachblock in the video picture and for each video picture in the videostream that is to be subject to subsampled decoding. Accordingly, thememory and bandwidth requirements of the video stream may besubstantially reduced with the satisfaction that the reduction comes atminimal cost in video quality assuming that the video stream is toultimately be subsample decoded. In one aspect of the invention, thefurther compressed video stream is sent to a subsample decoder where itis subsampled and presented on a display device.

Additional features and advantages of the invention will be set forth inthe description which follows, and in part will be obvious from thedescription, or may be learned by the practice of the invention. Thefeatures and advantages of the invention may be realized and obtained bymeans of the instruments and combinations particularly pointed out inthe appended claims. These and other features of the present inventionwill become more fully apparent from the following description andappended claims, or may be learned by the practice of the invention asset forth hereinafter.

BRIEF DESCRIPTION OF THE DRAWINGS

In order to describe the manner in which the above-recited and otheradvantages and features of the invention can be obtained, a moreparticular description of the invention briefly described above will berendered by reference to specific embodiments thereof, which areillustrated in the appended drawings. Understanding that these drawingsdepict only typical embodiments of the invention and are not thereforeto be considered to be limiting of its scope, the invention will bedescribed and explained with additional specificity and detail throughthe use of the accompanying drawings in which:

FIG. 1A illustrates a display order of an MPEG-2 video stream inaccordance with the prior art;

FIG. 1B illustrates a decode order of an MPEG-2 video stream inaccordance with the prior art;

FIG. 2A illustrates an encode sequence in accordance with MPEG-2 and inaccordance with the prior art;

FIG. 2B illustrates a decode sequence in accordance with MPEG-2 and inaccordance with the prior art;

FIG. 3 schematically illustrates a video network in which the principlesof the present invention may operate; and

FIG. 4 is a flowchart of a method for transcoding a video stream so thatthere is little, if any, loss in video quality after subsampling inaccordance with the present invention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

Subsampling is a process that reduces the dimensions of a video imagesuch as when the video stream is to be displayed in a reduced-sizepicture-in-picture display. The present invention extends to bothmethods and systems for reducing the size of the video stream withminimal, if any, effect on the video quality as displayed aftersubsampling. A video management system accesses a video stream byreceiving the video stream from a video channel, or by accessing amemory where the video stream is stored. Once the video managementsystem determines that only a reduced-size version of the video streamis ultimately to be displayed as when the video stream is to be subjectto subsampling, the video management system compresses each picture(e.g., frame or field) of the video frame. Although this compressionwould cause loss of picture quality if the picture were to be displayedin its full size, this compression is performed in such a manner thatthere is little, if any, loss in video quality as displayed aftersubsampling. Any loss in video quality would be primarily due tore-quantization, and finite-precision effects inherent in computerprocessing.

Embodiments within the scope of the present invention includecomputer-readable media for carrying or having computer-executableinstructions or data structures stored thereon. Such computer-readablemedia can be any available media that can be accessed by a generalpurpose or special purpose computer. By way of example, and notlimitation, such computer-readable media can comprise RAM, ROM, EEPROM,CD-ROM or other optical disk storage, magnetic disk storage or othermagnetic storage devices, or any other medium which can be used to carryor store desired program code means in the form of computer-executableinstructions or data structures and which can be accessed by a generalpurpose or special purpose computer.

When information is transferred or provided over a network or anothercommunications connection (either hardwired, wireless, or a combinationof hardwired or wireless) to a computer, the computer properly views theconnection as a computer-readable medium. Thus, any such connection isproperly termed a computer-readable medium. Combinations of the aboveshould also be included within the scope of computer-readable media.Computer-executable instructions comprise, for example, instructions anddata which cause a general purpose computer, special purpose computer,or special purpose processing device to perform a certain function orgroup of functions.

The precise operating environment in which the principles of the presentinvention are implemented is not important to the present invention. Theprinciples of the present invention may be implemented in any operatingenvironment that is able to implement the principles of the presentinvention. For example, given suitable software and/or adaptation,general-purpose computers, special-purpose computers or special purposeprocessing devices (whether now developed or to be developed in thefuture) might implement the principles of the present invention. Inaddition, the principles of the present invention may be implemented bysoftware, hardware, firmware or any combination thereof.

As will be described in further detail below, the principles of thepresent invention are most advantageous when the video processing inaccordance with the present invention is followed by subsampling. Theenvironment discussed below with respect to FIG. 3 illustrates just oneexample of an environment in which subsampling is performed and isprovided for illustrative purposes only, and not for purposes oflimiting the claims. One of ordinary skill in the art will easilyrecognize that the principles of the present invention may beimplemented in any environment where the video processed in accordancewith the present invention is to be subsampled.

FIG. 3 and the corresponding discussion provide a general description ofa network 300 in which the present invention may operate. The network300 includes a management system 310 that receives video input 301,performs appropriate processing on the video, and then distributes thevideo. The video is distributed either directly to a display device 311or else to a video node such as one of video nodes 320 through 323,where the video may be subject to further processing, including perhapssubsampling, before being distributed to the corresponding displaydevice 330 through 333. For illustrative purposes, four video nodes areshown although the management system 310 may work with other numbers ofvideo nodes. The management system 310 need not just perform videoprocessing. For example, the management system 310 may also communicatenon-video information with networks 303 over link 302 and process thenon-video information as well.

The video management system 310 includes a memory 341 that may store thecomputer-executable instructions described above, and a processor 342that is coupled to the memory 341 through, for example, a bus 343 so asto be able to execute the computer-executable instructions. The videomanagement system 310 also includes a video decoder 344 that decodesvideo in accordance with a video decoding standard such as, for example,MPEG. A transcoder 345 operates to reduce the memory and bandwidthrequirements of the video 302 and may do such by implementing theprinciples of the present invention described herein. If the videodecoder 344 and the transcoder 345 are implemented at least partially inhardware, the video decoder 344 and the transcoder 345 would be coupledto the bus 343 as shown in FIG. 3. However, as will be apparent to thoseof ordinary skill in the art, the principles of the present inventionmay be implemented by hardware, software, or a combination of hardwareand software.

While FIG. 3 and the corresponding discussion above provide a generaldescription of a suitable environment in which the invention may beimplemented, it will be appreciated that the features of the presentinvention disclosed herein may be practiced in association with avariety of different system configurations.

FIG. 4 illustrates a method 400 for transcoding a video stream so as toreduce the size of the video picture with minimal, if any, effect on thevideo quality after subsampling. In the context of FIG. 3, the videomanagement system 310 may perform this transcoding at a givensubsampling ratio and then provide the transcoded video stream to one ofthe video nodes (e.g., video node 320) over video network 300. The videonode 320 would then perform subsampled decoding using the samesubsampling ratio.

First, the video stream is accessed (act 401). Then, for at least one ofthe blocks in a video picture of the video stream, the size of theencoded block is reduced without substantially reducing image quality asmeasure after subsampling (act 402). Preferably, all of the blocks inall of the video pictures in the video stream are compressed when it isknown that the video stream is to be ultimately subject to subsampleddecoding. The processing of an already encoded video stream (or anyalready encoded data component for that matter) into a different encodedvideo stream is often referred to as “transcoding” since the videostream is moved from one encoded state to another. The compression ofthe encoded video stream to generated a more compressed video streamwith minimal, if any, loss in video quality after subsampling is oneexample of transcoding and will be referred to herein as “subsampledtranscoding” although subsampled decoding is still needed after thesubsampled transcoding in order to display the reduced size image. Inthe context of FIG. 3, for example, this subsampled transcoding may beperformed by transcoder 345.

In order to generate the reduced size blocks, the block of pixels isrepresented as a matrix (act 403). The block of pixels may either berepresented by a “spatial domain” matrix or by a “transform domain”matrix. A spatial domain matrix of a block of pixels means that theelement values of the matrix are specific pixel values that are laid outspatially in the matrix according to the position of the correspondingpixel in the block. Thus, the element in row 3, column 2 of the spatialdomain matrix represents a pixel value corresponding to a pixel in row3, column 2 of the block of pixels. A transform domain matrix of a blockof pixels is less intuitive and is represented by performing a transformon the spatial domain matrix. Each element in the transform domainmatrix represents a discrete transform relationship between the pixelvalues in the spatial domain matrix. For example, if the transformdomain matrix is a frequency domain matrix, one common discretefrequency relationship is defined by the well-known Discrete CosineTransform (DCT) operation. Before describing a suitablepre-multiplication matrix and post-multiplication matrix that aresuitable for acts 404 and 405, respectively, the mathematicalrelationship between transform domain matrices and spatial domainmatrices will now be briefly described followed by a mathematicaldescription of how subsampling typically occurs.

A transform domain matrix A may be generated by performingpre-multiplication and post-multiplication on a corresponding spatialdomain matrix P. This operation is represented in matrix form by thefollowing equation 1:

1:A=D×P×E  (1)where,

-   -   P is the spatial domain matrix corresponding to the transform        domain matrix A;    -   A is the transform domain matrix corresponding to the spatial        domain matrix P;    -   D is the transform matrix for the vertical direction; and    -   E is the transform matrix for the horizontal direction.

If the spatial domain matrix P is, for example, an 8-by-8 matrix whereeach element represents a pixel component, the matrices D, E and A arealso 8-by-8 matrices. There is no requirement that the matrices D and Ebe unitary or symmetric. Also there is no requirement that the D and Erepresent the same transform. In one case, D could represent a DiscreteCosine Transform (DCT) matrix in the vertical direction, while Erepresents D transpose (i.e., the DCT matrix in the horizontaldirection). However, in another example D could represent a wavelettransform matrix in the vertical direction, while D represents the DCTmatrix in the horizontal direction.

Conversely, the spatial domain matrix P may be generated from atransform domain matrix by performing an inverse matrix transform on thetransform domain matrix. This inverse operation is represented in matrixform by the following equation 2:P=D ⁻¹ ×A×E ⁻¹  (2)

Subsampling of the spatial domain matrix P occurs by pre-multiplying thematrix P by a subsampling matrix that defines the subsampling in onedirection such as when performing horizontal subsampling. The resultingsubsampled matrix may then be post-multiplied by the transpose ofanother subsampling matrix that defines the subsampling in asubstantially perpendicular direction as when performing verticalsubsampling. This subsampling is performed on the spatial domain matrixP as illustrated by the following equation 3:p=S×P×T′  (3)where,

-   -   p is the subsampled spatial domain matrix of the spatial domain        matrix P;    -   S is the subsampling matrix that is used for horizontal        subsampling; and    -   T′ is the transpose of the subsampling matrix T that is used for        vertical subsampling.

Rewriting equation 3 by substituting the value of matrix P from equation2 results in the following equation 4:p=S×D ⁻¹ ×A×E ⁻¹ ×T′  (4)

Each of these matrices may conceptually be split into four separatequadrants based on the subsample size. For example, the matrix A may berewritten as the following equation 5: $\begin{matrix}{A = \begin{bmatrix}A_{TL} & A_{TR} \\A_{BL} & A_{BR}\end{bmatrix}} & (5)\end{matrix}$where,

-   -   A_(TL) is a matrix component having a size that is proportional        to the matrix A by the same ratio as the proportion of the        subsampled picture to the original picture;    -   A_(TR) is a matrix component that resides to the right of the        matrix A_(TL);    -   A_(BL) is a matrix component that resides below the matrix        A_(TL); and    -   A_(BR) is a matrix component that resides below the matrix        A_(TR) and to the right of the matrix A_(BL).

Similarly, the top two components of the matrix may be combined and thebottom two components may be combined so that the matrix A is defined asin the following equation 6: $\begin{matrix}{A = \begin{bmatrix}A_{T} \\A_{B}\end{bmatrix}} & (6)\end{matrix}$where,

-   -   A_(T) is a matrix component defined by the combination of A_(TL)        and A_(TR); and    -   A_(B) is a matrix component defined by the combination of A_(BL)        and A_(BR).

For instance, if the matrix A is an 8 row by 8 column matrix, and thesubsampling cuts each dimension size (horizontal and vertical) in half,the matrix component ATL would be a 4 row by 4 column matrix.Consequently, the other matrix components ATR, ABL, and ABR would alsobe 4 row by 4 column matrices. In this case, the matrix components ATand AB would each be 4 rows by 8 columns. This subsample ratio is usedas an example in the following description although the presentinvention works with other subsampling ratios. For example, if thepicture were subsampled by 75%, each 8 row by 8 column matrix would bereduced to a mere 2 row by 2 column matrix. In this latter case, thematrix component ATL would be a 2 row by 2 column matrix. Consequently,the matrix components ATR would be a 2 row by 6 column matrix, thematrix component ABL would be a 6 row by 2 column matrix, and the matrixcomponent ABR would be a 6 row by 6 column matrix. In this case, thematrix component AT would be two rows by eight columns and the matrixcomponent AB would be 6 rows by 8 columns. Subsampling may also occur injust one direction, horizontal or vertical, with no subsamplingoccurring in the other direction.

Thus, the size of the matrix components is important for the transcoderto know when performing the subsample transcoding in accordance with thepresent invention since the subsampling ratio used to perform subsampletranscoding by the transcoder 345 should be the same as the subsamplingratio used to perform subsampled decoding at the video node 320. Thisknowledge may be inferred by the transcoder 440. For example, if thevideo stream corresponds to the reduced-size image of apicture-in-picture display, the reduced size image may always be acertain size (e.g., half the size for each dimension). Thus, thetranscoder 440 may infer that if subsampled decoding is to occur at all,it is at a subsampling ratio of 50% in each direction.

Referring to FIG. 3, the video node (e.g., video node 320) that is todisplay the reduced size image informs the management system 310 vianetwork 300 the identity of any channel for which a reduced size videoimage is desired. If the transcoder 345 cannot accurately infer thesubsampling ratio (e.g., a user may be able to adjust the size of thepicture-in-picture image), the video node 320 also informs the videomanagement system of the appropriate subsampling ratio that is to beperformed at the video node.

In accordance with the principles of the present invention, thetransform domain matrix A is converted into a matrix a that has zerovalues in all but its upper left component aTL. Specifically, matrix amay be represented by the following equation 7. $\begin{matrix}{a = \begin{bmatrix}a_{TL} & Z \\Z & Z\end{bmatrix}} & (7)\end{matrix}$where,

-   -   Z represents matrix components having zero values for all        elements.

Since the matrix a has many zero values, coding methods such as Huffmanvariable length coding reduce the coded representation of the matrix asignificantly as compared to the coded representation of the matrix A.Thus, the size of the coded video stream is significantly reduced whenconverting matrix A to the matrix a for each block in each picture ofthe video stream.

In accordance with the principles of the present invention, the matrix Ais converted into the matrix a in such a manner that subsampled decodingof the matrix a results in the same pixel block (i.e., matrix p definedby equation 4) as subsampled decoding the matrix A. Specifically, thefollowing equation 8 holds true:S×D ⁻¹ ×A×E ⁻¹ ×T′=S×D ⁻¹ ×a×E ⁻¹ ×T′  (8)

As mentioned above, the matrix a only has the potential for non-zeroelements in its upper left matrix component aTL. Thus, once onedetermines what the matrix component aTL should be, one has alsodetermined what the matrix a should be. The inventors have discoveredthat an appropriate matrix component aTL that will cause equation 8 tobe satisfied so that subsampled decoding of the matrix a results insubstantially the same picture as subsampled decoding of the matrix A isdefined by the following equation 9:a _(TL)=(m 1)⁻¹ ×S×P×T′×(n 1)⁻¹  (9)

In equation 9, the matrix (m1)⁻¹×S represents an example of thepre-multiplication matrix that the block matrix P is pre-multiplied byin act 404 of FIG. 4. The matrix T′×(n1)⁻¹ represents an example of apost-multiplication matrix that the block matrix P is post-multiplied byin act 405 of FIG. 4.

In equation 9, the matrix P represents the spatial domain representationof a block of pixels and, in a typical example, is an 8 row by 8 columnmatrix.

The matrix S is the vertical subsampling matrix. For example, if eachdimension of the picture is cut in half when subsampling, and the blockP has 8 rows, the matrix S would be a 4 row by 8 column matrix.

The matrix T′ is the transpose of the matrix T. The matrix T is thehorizontal subsampling matrix. For example, if each dimension of thepicture is cut in half when subsampling, and the block P has 8 columns,the matrix T′ would be an 8 row by 4 column matrix.

The matrix (m1)−1 is the multiplication inverse of matrix m1 such that(m1)−1×m1=I where “I” is the identity matrix. Matrix m1 equalsS×(D−1)left, where (D−1)left is the left portion of the inverse of D. Asan illustrative but non-limiting example of the dimension of m1, if S isa four row by eight column matrix and (D−1)left is an eight row by fourcolumn matrix, the matrix m1 and the matrix (m1)−1 are both four row byfour column matrices.

The matrix (n1)−1 is the multiplication inverse of matrix n1 such that(n1)−1×n1=I where “I” is the identity matrix. Matrix n1 equals(E−1)top×T′. As an illustrative but non-limiting example of the size ofn1, if (E−1)top is a four row by eight column matrix and T′ is an eightrow by four column matrix, the matrix n1 and the matrix (n1)−1 are bothfour row by four column matrices.

The dimension of the resulting pre-multiplication matrix (m1)−1×S bywhich the matrix P is pre-multiplied according to equation 9 is obtainedas follows. The number of rows in this matrix is equal to the number ofrows in S and the number of columns in this matrix is equal to thenumber of rows in P (the latter typically being 8).

The dimension of the resulting post-multiplication matrix T′×(n1)⁻¹ bywhich the matrix P is post-multiplied according to equation 9 isobtained as follows. The number of rows in this matrix is equal to thenumber of columns in P (which is typically 8), and the number of columnsis equal to the number of rows in T.

To continue with the illustrative but non-limiting example of thedimensions of the matrices involved, in this example, equation 9 resultsin the pre-multiplication of an eight row by eight column matrix by afour row by eight column pre-multiplication matrix, and in thepost-multiplication of the eight row by eight column matrix by an eightrow by four column pre-multiplication matrix. In this example, theresult is a four row by four column matrix that constitutes thepotential non-zero values of the eight row by eight column matrix a.

Referring to FIG. 4, after the subsample transcoding is completed foreach block, variable length coding is performed on the blocks and thenthe transcoded video is either stored for future subsampled decoding, orthe transcoded video is provided to the subsample decoder (act 406). Inthe context of FIG. 3, the video management system 310 may either storethe video stream in memory 341, or else may provide the video stream toanother component external to the video management system 310.

Since the transcoded video stream is smaller after variable lengthcoding than the original encoded video stream, less memory is requiredto store the video stream if the video stream is stored. Also, lessnetwork bandwidth is required to transmit the video stream if the videostream is transmitted over the network. According, the memory andnetwork bandwidth needed to handle the video stream are reduced. Thesubsample transcoding described above would result in a loss of imagequality if subsampled decoding was not to occur. However, if it is knownthat the video stream is to ultimately be subsample decoded, the videostream may be subsampled transcoded in accordance with the presentinvention with the assurance that the subsampled transcoding will resultin no lost image quality after subsampled decoding. Accordingly,although some additional processing is required to perform thesubsampled transcoding, the principles of the present invention allowfor reduced memory and bandwidth requirements with no cost in terms ofloss of video quality after subsampled transcoding.

The present invention may be embodied in other specific forms withoutdeparting from its spirit or essential characteristics. The describedembodiments are to be considered in all respects only as illustrativeand not restrictive. The scope of the invention is, therefore, indicatedby the appended claims rather than by the foregoing description. Allchanges which come within the meaning and range of equivalency of theclaims are to be embraced within their scope.

1. In a video management system configured to receive a video streamcontaining one or more video pictures that are each divided into blocks,wherein the video management system is to provide a representation ofthe one or more video pictures to a subsample decoder for subsampling, amethod of reducing the size of the one or more frames with minimal, ifany, effect on the video quality generated from the one or more framesafter subsampling, the method comprising the following: accessing avideo picture that is to be provided to a video node having a subsampledecoder; obtaining information specifying a subsampling ratio to beperformed by the subsample decoder; and in response to the obtainedinformation, for at least one block of the video picture, performingsubsample transcoding of the at least one block, thereby reducing thesize of the at least one block to generate at least one reduced sizeblock in such a way that subsampled decoding the at least one reducedsize block results in substantially the same reduced size image assubsampled decoding the at least one block prior to the subsampletranscoding.
 2. The method in accordance with claim 1, wherein themethod further comprises the following: providing the video picture withits one or more reduced size blocks to the subsample decoder.
 3. Themethod in accordance with claim 1, wherein the method further comprisesthe following: subsampled decoding the video picture.
 4. The method inaccordance with claim 3, wherein the method further comprises thefollowing: displaying the subsample decoded video picture on a displaydevice.
 5. The method in accordance with claim 3, further comprising thefollowing: displaying the subsample decoded video picture on a displaydevice as a reduced sized picture of a picture-in-picture display. 6.The method in accordance with claim 1, wherein the act of reducing thesize of the block to generate a reduced size block comprising thefollowing: representing the block as a matrix; pre-multiplying the blockmatrix by a pre-multiplication matrix, the pre-multiplication matrixgenerated from a first subsample matrix that represents the subsampleddecoding in a first direction; and post-multiplying the block matrix bya post-multiplication matrix generated from a second subsample matrixthat represents the subsampled decoding in a second direction that issubstantially perpendicular to the first direction.
 7. The method inaccordance with claim 6, further comprising: receiving an indicationover a network that the subsample decoder is to operate on the videostream.
 8. The method in accordance with claim 6, further comprising:receiving the first and second subsample matrix over a network from thesubsample decoder.
 9. The method in accordance with claim 6, wherein thefirst subsample matrix is a vertical subsample matrix that representsthe vertical subsampling that is to be performed by the subsampledecoder, and wherein the second subsample matrix is a horizontalsubsample matrix that represents the horizontal subsampling that is tobe performed by the subsample decoder, the method further comprising thefollowing: generating the pre-multiplication matrix by performing thefollowing: determining a first inverse matrix that represents themultiplication inverse of a matrix that results from the multiplicationof a first transform matrix times the vertical subsampling matrix; andmultiplying the first inverse matrix by the vertical subsampling matrixto generate the pre-multiplication matrix.
 10. The method in accordancewith claim 9, wherein determining a first inverse matrix that representsthe multiplication inverse of a matrix that results from themultiplication of a first transform matrix times the verticalsubsampling matrix comprises the following: determining a first inversematrix that represents the multiplication inverse of a matrix thatresults from the multiplication of a vertical discrete cosign transformmatrix times the vertical subsampling matrix.
 11. The method inaccordance with claim 9, wherein determining a first inverse matrix thatrepresents the multiplication inverse of a matrix that results from themultiplication of a first transform matrix times the first subsamplingmatrix comprises the following: determining a first inverse matrix thatrepresents the multiplication inverse of a matrix that results from themultiplication of a vertical wavelet transform matrix times the firstsubsampling matrix.
 12. The method in accordance with claim 9, furthercomprising the following: generating the post-multiplication matrix byperforming the following: determining a second inverse matrix thatrepresents the multiplication inverse of a matrix that results from themultiplication of a second transform matrix times the transpose of thehorizontal subsampling matrix; and multiplying the transpose of thehorizontal subsampling matrix by the second inverse matrix to generatethe post-multiplication matrix.
 13. The method in accordance with claim12, wherein determining a second inverse matrix that represents themultiplication inverse of a matrix that results from the multiplicationof a second transform matrix times the horizontal subsampling matrixcomprises the following: determining a second inverse matrix thatrepresents the multiplication inverse of a matrix that results from themultiplication of a horizontal discrete cosign transform matrix timesthe horizontal subsampling matrix.
 14. The method in accordance withclaim 12, wherein determining a second inverse matrix that representsthe multiplication inverse of a matrix that results from themultiplication of a second transform matrix times the horizontalsubsampling matrix comprises the following: determining a second inversematrix that represents the multiplication inverse of a matrix thatresults from the multiplication of a horizontal wavelet transform matrixtimes the horizontal subsampling matrix.
 15. The method in accordancewith claim 12, further comprising the following: horizontal subsamplingthe video picture using the horizontal subsampling matrix; and verticalsubsampling the video picture using the vertical subsampling matrix. 16.The method in accordance with claim 1, wherein the at least one block ofthe video picture comprises all the blocks in the video picture.
 17. Themethod in accordance with claim 1, wherein the act of accessing a videopicture that is to be subsampled comprises the following: accessing avideo frame that is to be subsampled.
 18. The method in accordance withclaim 1, wherein the act of accessing a video picture that is to besubsampled comprises the following: accessing a video frame that is tobe subsampled.
 19. A method as recited in claim 1, wherein theinformation specifying the subsampling ratio is provided by the videonode.
 20. A method as recited in claim 19, wherein the informationspecifying the subsampling ratio is provided upon determining that thesubsampling ratio cannot be inferred.
 21. A method as recited in claim1, wherein the information specifying the subsampling ratio correspondsto a reduced-size in which the video picture will be displayed by thevideo node.
 22. A method as recited in claim 1, wherein subsampleddecoding of the at least one block causes the video image to be renderedwithout image loss that would otherwise occur if the image were to berendered without the subsampled decoding.
 23. A computer program productcomprising one or more computer-readable media havingcomputer-executable instructions for implementing a method in a videomanagement system configured to receive a video stream containing one ormore video pictures that are each divided into blocks, wherein the videomanagement system is to provide a representation of the one or morevideo pictures to a subsample decoder for subsampling, wherein themethod is for reducing the size of the one or more frames with minimal,if any, effect on the video quality generated from the one or moreframes after subsampling, the method comprising the following: accessinga video picture that is to be provided to a video node having asubsample decoder; obtaining information specifying a subsampling ratioto be performed by the subsample decoder; and in response to theobtained information, for at least one block of the video picture,performing subsample transcoding of the at least one block, therebyreducing the size of the at least one block to generate at least onereduced size block in such a way that subsampled decoding the at leastone reduced size block results in substantially the same reduced sizeimage as subsampled decoding the at least one block prior to thesubsample transcoding.
 24. A video management system configured toreceive a video stream containing one or more video pictures that areeach divided into blocks, wherein the video management system is toprovide a representation of the one or more video pictures to asubsample decoder for subsampling, the video management systemcomprising the following: means for accessing a video picture to beprovided to a video node having a subsample decoder; means fordetermining a subsampling ratio to be performed by the subsample decoderprior to providing the video picture to the video node; and means fortranscoding the video picture such that subsampled decoding the reducedsize picture results in substantially the same reduced size image assubsampled decoding the original video picture.